Duality of Maximum Entropy and Minimum Divergence
نویسندگان
چکیده
منابع مشابه
Duality of Maximum Entropy and Minimum Divergence
We discuss a special class of generalized divergence measures by the use of generator functions. Any divergence measure in the class is separated into the difference between cross and diagonal entropy. The diagonal entropy measure in the class associates with a model of maximum entropy distributions; the divergence measure leads to statistical estimation via minimization, for arbitrarily giving...
متن کاملA Maximum Entropy/minimum Divergence Translation Model
I present empirical comparisons between a standard statistical translation model and an equivalent Maximum Entropy/Minimum Divergence (MEMD) model, using several diierent methods for automatic feature selection. Results show that the MEMD model signiicantly outperforms the standard model in test corpus perplexity, even though it has far fewer parameters.
متن کاملA Maximum Entropy/Minimum Divergence Translation Model
I present empirical comparisons between a linear combination of standard statistical language and translation models and an equivalent Maximum Entropy/Minimum Divergence (MEMD) model, using several diierent methods for automatic feature selection. The MEMD model signiicantly outperforms the standard model in test corpus per-plexity, even though it has far fewer parameters.
متن کاملFast computation of maximum entropy / minimum divergence feature gain
Maximum entropy / minimum divergence modeling is a powerful technique for constructing probability models, which has been applied to a wide variety of problems in natural language processing. A maximum entropy / minimum divergence (memd) model is built from a base model, and a set of feature functions, also called simply features, whose empirical expectations on some training corpus are known. ...
متن کامل3 : Maximum Likelihood / Maximum Entropy Duality
In the previous lecture we defined the principle of Maximum Likelihood (ML): suppose we have random variables X1, ..., Xn form a random sample from a discrete distribution whose joint probability distribution is P (x | φ) where x = (x1, ..., xn) is a vector in the sample and φ is a parameter from some parameter space (which could be a discrete set of values — say class membership). When P (x | ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Entropy
سال: 2014
ISSN: 1099-4300
DOI: 10.3390/e16073552